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Appellant: George V. Paul et al. 
Serial Na: 09/896,150 
Filing Date: June 29, 2001 




Group Art Unit: 3713 
Examiner: Scott E. Jones 



For: REAL-TIME HEAD TRACKING FOR COMPUTER GAMES AND OTHER 
APPLICATIONS 



APPELLANTS' CORRECTED BRIEF UNDER 37 CFR §1.192 



Mail Stop Appeal Brief 
Commissioner for Patents 
PO Box 1450 

Alexandria, VA 22313-1450 
Dear Sir: 

In response to the Notification of Non-Compliance dated November 8, 2004, Appellants' submit 
their Corrected Appeal Brief in triplicate. 

L Real Party in Interest 

The real parties and interest in this case are inventors George Paul, Glenn Beach, Charles Cohen, 
and Charles Jacobus, Applicants and Appellants. 

II. Related Appeals and Interferences 

There are no appeals or interferences which will directly affect or be directly affected by or have 
a bearing on the Board's decision in the pending appeal. 

III. Status of Claims 

The present application was filed with 13 claims. Claim 11 was canceled in August 2003. 
Claims 1-10 and 12-13 remain pending and are under appeal. 
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IV. Status of Amendments Filed Subsequent 
Final Rejection 

An after-final amendment is submitted herewith to correct the priority claim. 



V. Concise Summary of the Invention 

This invention resides in a real-time computer vision system capable of tracking moving objects 
in a scene. Unlike current search and locate algorithms, the subject algorithm uses a target location 
technique which does not involve search (Specification, page 4, lines 2-4). The system tracks objects 
based on the color, motion and shape of the object in the image. The tracking algorithm uses a unique 
color matching technique which uses minimal computation. This color matching function is used to 
compute three measures of the target's probable location based on the target color, shape and motion. It 
then computes the most probable location of the target using a weighting technique (Specification, page 
4, lines 4-9). These techniques make the invention very computationally efficient also makes it robust 
to noise, occlusion and rapid motion of the target (Specification, page 4, lines 9-10). 

The imaging hardware of the real-time object tracking system includes a color camera, a frame 
grabber, and a personal computer. The software includes low-level image grabbing software and the 
tracking algorithm (Specification, page 4, lines 1 1-13). Once the application is running, a graphical user 
interface displays the live image from the color camera on the computer screen. The operator can then 
use the mouse to click on the hand in the image to select a target for tracking (Specification, page 4, 
lines 13-15). The system will then keep track of the moving target in the scene in real-time. As such, 
the system finds utility in a wide range of applications, including head tracking for game control 
(Specification, page 4, lines 15-22). 

VI. Concise Statement of Issues Presented 
For Review 

1. Are claims 1-10 and 12-13 unpatentable under 35 U.S.C. §103(a) over U.S. Patent No. 
6,075,557 to Holliman et al. in view of the Birchfield technical paper? 
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VII. Grouping of Claims for Each Ground of 
Rejection Which Appellants Contend 



Group I: Claims 1-10, 12 and 13, wherein claims 2-10, 12 and 13 stand or fall with claim 1. 
VIII. Argument 

Group I - Claims 1-10, 12 and 13, wherein claims 2-10, 12 and 13 stand or fall with claim 1. 

Claim 1 stands rejected under 35 U.S.C. § 103(a) as being unpatentable over Holliman et al., 
U.S. Patent No. 6,075,557, in view of Birchfield. Among other elements, claim 1 includes the 
limitation of "providing a weighted average of color to compute the location of the user's head based 
upon color alone." The Examiner concedes that Holliman et al. are silent with respect to this aspect, but 
argues that Birchfield address this deficiency citing page 2, column 2 to page 3, column of the 
Birchfield paper. The Examiner is incorrect. 

Birchfield teaches a method for object tracking that combines the output of two different 
modules: one that matches the intensity gradients along the object's boundary, and one that matches a 
color histogram of the object' s interior. The latter, color module considers the color of a human head 
to be complex and being at the very least bimodal due to the usually contrasting colors of the skin and 
hair. Birchfield consequently uses a color histogram in its method for target tracking because of the 
ability of the color histogram to inclusively capture complex, multi -modal patterns of color of the kind. 

The Birchfield reference goes on to teach the computation of a histogram intersection between a 
model histogram and an image histogram at each hypothesized location of the user's head. The paper 
goes on to explain that the power of the histogram intersection results from the MIN( ) function, which 
matches no more image pixels of a certain color that are present in the model histogram. Thus, for 
example, the measure of accuracy for object tracking is more satisfied within a region containing both 
facial and hair color than a region containing all facial color. 

It is well-settled that if a proposed combination of prior-art references would change the 
principle of operation of the prior art invention being modified, then the teachings of the references are 
not sufficient to render the claims prima facie obvious. In re Ratti, 270 F.2d 810, 123 USPQ 349 
(CCPA 1959). Such is the case at bar. Birchfield teaches using the MIN( ) function to determine the 
histogram intersection between a model histogram and an image histogram for object tracking based on 
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<° color alone. Appellants claim the use of a weighted average of color to compute the location of a user's 
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~ head based upon color alone. 

Furthermore, in rejecting claims under 35 U.S.C. §103, the Examiner must provide a reason 

why one having ordinary skill in the pertinent art would have been led to combine the cited references to 

f arrive at Appellants' claimed invention. There must be something in the prior art that suggests the 
o 

5 „ proposed combination, other than the hindsight gained from knowledge that the inventor choose to 
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g combine these particular things in this particular way. Uniroyal Inc. v. Rudkin- Wiley Corp. , 837 F.2d 
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1 1044, 1051, 5 USPQ2d 1434, 1438 (Fed. Cir. 1988). The Examiner is also required to make specific 
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USPQ2d 1614, 1617-19 (Fed. Cir. 1999). 
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I The Examiner has not met these requisite standards. As for motivation to combine, the 
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a Examiner argues that the combination "provides for a robust head tracking system that is accurate 
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| enough to accurately control the camera's pan, tilt and zoom for long periods of time in order to keep 
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I the user's head in the field of view at the desired size." However, these are considerations not found in 

Q 

o Appellants' claims, or in the prior art for that matter. For example, Appellants can find no evidence in 
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| the primary reference of Holliman et al. regarding panning, tilting or zooming. As such there can be no 
qj motivation to improve that which does not exist. Appellants do not claim, nor does the prior art of 
| record recite a need for "accurate control ... for long periods of time in order to keep the user's head in 

g the field of view at the desired size." 
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John G. Pos? 
Reg. No. 3: 

Gifford, Krass, Groh, Sprinkle, 
Anderson & Citkowski, P.C. 
280 N. Old Woodward, Suite 400 
Birmingham, MI 48009 
(734) 913-9300 
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1. A method controlling a computer game, comprising the steps of: 
5 imaging a sequence of scenes including the head of a user of the computer; 

5 comparing visual characteristics from a portion of a scene to a center of said portion of a scene 

X 

I to determine movement of the user's head within the scene wherein at least one of the visual 
| characteristics is color; 

o 

° providing a weighted average of color to compute the location of the user's head based upon 

^ color alone; and 

D 

| controlling the game in accordance with the movements. 

a 

DC 

I 

§ 2. The method of claim 1, wherein the visual characteristics include shape or location. 
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o 4. The method of claim 3, further including the step of modeling of the dynamic 
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5. The method of claim 1, further including the step of initiating head tracking through a 



j graphical user interface. 
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o 6. The method of claim 5, wherein the graphical user interface provides a bounding box 
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| displayed in a screen to assist in targeting the user's head. 
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g 7. The method of claim 2, further comprising the step of enabling a match in color despite 



0 the differences arising from lighting and shadows. 
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§ 13. The method of claim 1, further including the step of continuing to track the user's head 
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i when moving in front of or behind a similarly colored object in the scene. 



predetermined closeness of color as an estimate of target shape. 
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Appellant: George V. Paul et al. 

Serial No.: 09/896,150 Group Art Unit: 3713 

Filing Date: June 29, 2001 Examiner: Scott E. Jones 

For: REAL-TIME HEAD TRACKING FOR COMPUTER GAMES AND OTHER 
APPLICATIONS 



CERTIFICATE OF MAILING UNDER 37 CFR 1.8(a) 

Mail Stop AF 
Commissioner for Patents 
PO Box 1450 

Alexandria, VA 22313-1450 
Dear Sir: 

I hereby certify that the attached correspondence comprising: 

Corrected Appeal Brief (in triplicate) 
Postcard 



is being deposited with the United States Postal Service as first class mail in an envelope addressed 
to: 

Mail Stop AF 
Commissioner for Patents 
PO Box 1450 
Alexandria, VA 22313-1450 



On Dec. 8, 2004 . 



Slieryl L. Hammer 



